Cartoon-style drawing and animation have over time developed a rich language of non-photorealistic stylizations. The ability to rotate a two-dimensional (2D) cartoon and view it from any angle has many applications, both in aiding animation and in enabling cartoon objects to be placed in an interactive three-dimensional (3D) environment in which the user controls the viewpoint. Previously, this has been achieved by constructing a 3D model of the cartoon and rendering it in a non-photorealistic way so as to resemble a cartoon. However, generating a 3D model is time-consuming and may be processing intensive. In addition, many stylistic elements of 2D drawings cannot be adequately reproduced in a 3D model, as the 2D drawings may not correspond to any real 3D shapes.
Non-photorealistic rendering techniques have been proposed to make a 3D model resemble a hand-drawn cartoon, such as cel-shaded lighting or exaggerated, 2D silhouettes. However, rendering from a single 3D model cannot account for cartoons that have mutually inconsistent appearances in different views—for example, no matter the viewing angle, Bugs Bunny's ears are always facing the camera. View-dependent geometry addresses this limitation by providing multiple different models of an object, each associated with a single perspective, and rendering intermediate perspectives using a 3D model that is an interpolation of the models for nearby orientations. This approach, while more flexible, requires additional 3D modeling, and retains the essential 3D nature and appearance of the model.
Di Fiore et al. proposed an approach in which a set of hand-drawn views are interpolated to generate renderings at intermediate views in their 2001 paper, “Automatic In-betweening in Computer Assisted Animation by Exploiting 2.5D Modelling Techniques.” In their approach, either a 3D skeleton must be manually created by the artist, or Z-ordering must be manually specified at each view. In the latter case, interpolation of shapes is simple linear interpolation. These limitations make di Fiore's approach unsuitable for generating a large number of rotated views of 2D cartoons.
Several methods have been proposed for the interpolation of 2D shapes, including solving the problem of vertex correspondence between two shapes by using a physically-based approach, in which strokes are likened to lengths of wire and a dynamic programming algorithm is used to minimize the amount of bending and stretching the wire undergoes in blending between two shapes. Other methods address the problem of vertex paths, by formulating as-rigid-as-possible shape interpolation, in which paths for vertices are found that keep the volume of the interior of the stroke at intermediate points as rigid as possible. Further methods introduced an as-rigid-as-possible approach to interactive shape manipulation.
While these interpolation methods are purely 2D, there is a form of 2D image interpolation, known as “View Morphing,” that can mimic 3D rotations between two views. However, View Morphing's approach involves warping images where corresponding 2D points represent the same 3D point. In the case of cartoons, strokes often represent silhouettes of a shape, and corresponding 2D points therefore often do not represent a single 3D point.
Related work describes mapping 2D vector art “key frames” to positions in 3D, and allowing a user to explore the space of possible interpolations by manipulating a 3D point. The nearest keyframes to the 3D point are interpolated and the result is presented to the user. Alternative work limits the configuration space of interpolations between drawings to the subset that generates plausible images. The user manipulates the image to explore that subset of the state space without explicit knowledge of how the state space is laid out. Still other work introduces a new type of stroke for vector art which, amongst other things, can map different stroke lengths to different 2D drawings, which may be interpolated for intermediate lengths.
Several techniques aim to blend the distinction between 2D and 3D drawings. One approach includes drawing 2D strokes onto a plane in a 3D scene. These strokes are interpreted as indicating silhouettes or contours of 3D objects, and correspondingly deform and fade into and out of visibility as the camera rotates around the scene. Another approach uses 2D strokes that are again embedded in 3D space, but in this case they represent silhouettes of a shape and are explicitly associated with one another, so as the camera rotates, just the appropriate silhouette can be displayed.
Generating a new view of a cartoon may require determination of the strokes' Z-ordering in addition to the determination of the strokes' shapes. Several techniques have been proposed that seek to combine 2D drawings with depth information. One uses a new interface to determine Z-ordering of the strokes in a cartoon using sparse depth inequalities. Another layers elements in a flat image with local layering constraints, rather than global ones, effectively allowing depths that vary across the object. Reversing the challenge, a third automatically composes a 2D layered image from a 3D scene. Finally, a variety of sketch-based modeling approaches have been proposed, which aim to generate explicit 3D models from 2D drawings.